Visualization techniques are a critical means of shedding light on the behavior of metaheuristic numerical optimization algorithms. Conventional methods such as convergence analysis, trajectory visualizations, and fitness landscape analysis provide valuable insights into aspects like convergence speed, diversity, and solution quality. However, these approaches often fail to capture the structural dynamics of the search process, particularly in high-dimensional or complex spaces. Existing methods rarely address the location of the solution candidates in the search space, which can reveal crucial information about the exploratory and exploitative strategies of an algorithm. We propose ClustOpt, a novel representation and visualization methodology for metaheuristic numerical population-based optimization algorithms, that focuses on clustering solution candidates explored by optimization algorithms.

Extending a recent suggestion to generate new instances for numerical black-box optimization benchmarking by interpolating pairs of the well-established BBOB functions from the COmparing COntinuous Optimizers (COCO) platform, we propose in this work a further generalization that allows multiple affine combinations of the original instances and arbitrarily chosen locations of the global optima. We demonstrate that the MA-BBOB generator can help fill the instance space, while overall patterns in algorithm performance are preserved. By combining the landscape features of the problems with the performance data, we pose the question of whether these features are as useful for algorithm selection as previous studies suggested. MA-BBOB is built on the publicly available IOHprofiler platform, which facilitates standardized experimentation routines, provides access to the interactive IOHanalyzer module for performance analysis and visualization, and enables comparisons with the rich and growing data collection available for the (MA-)BBOB functions.

Hyperparameter optimization (HPO) is a key component of machine learning models for achieving peak predictive performance. While numerous methods and algorithms for HPO have been proposed over the last years, little progress has been made in illuminating and examining the actual structure of these black-box optimization problems. Exploratory landscape analysis (ELA) subsumes a set of techniques that can be used to gain knowledge about properties of unknown optimization problems. In this paper, we evaluate the performance of five different black-box optimizers on 30 HPO problems, which consist of two-, three- and five-dimensional continuous search spaces of the XGBoost learner trained on 10 different data sets. This is contrasted with the performance of the same optimizers evaluated on 360 problem instances from the black-box optimization benchmark (BBOB). We then compute ELA features on the HPO and BBOB problems and examine similarities and differences. A cluster analysis of the HPO and BBOB problems in ELA feature space allows us to identify how the HPO problems compare to the BBOB problems on a structural meta-level. We identify a subset of BBOB problems that are close to the HPO problems in ELA feature space and show that optimizer performance is comparably similar on these two sets of benchmark problems. We highlight open challenges of ELA for HPO and discuss potential directions of future research and applications.

LTHOUGH the Algorithm Selection Problem (ASP, [1]) has been introduced more than four decades ago, there only exist few works (e.g., [2], [3]), which perform algorithm selection in the field of continuous optimization. Independent of the underlying domain, the goal of the ASP can be described as follows: given a set of optimization algorithms A, often denoted algorithm portfolio, and a set of problem instances I, one wants to find a model m: I A that selects the best algorithm A A from the portfolio for an unseen problem instance I I. Albeit there already exists a plethora of optimization algorithms - even when only considering singleobjective, continuous optimization problems - none of them can be considered to be superior to all the other ones across all optimization problems. Hence, it is very desirable to find a sophisticated selection mechanism, which automatically picks the portfolio's best solver for a given problem. Within other optimization domains, such as the well-known Travelling Salesperson Problem, feature-based algorithm selectors have already shown their capability of outperforming the respective state-of-the-art optimization algorithm(s) by combining machine learning techniques and problem dependent features [4], [5].